Interrater Reliability of Standardized Actors Versus Nonactors in a Simulation Based Assessment of Interprofessional Collaboration.
نویسندگان
چکیده
INTRODUCTION There is a need for reliable and practical interprofessional simulations that measure collaborative practice in outpatient/community scenarios where most health care takes place. The authors applied generalizability theory to examine reliability in an ambulatory care scenario using the following 2 trained observer groups: standardized patient (SP, actor) raters and those who received rater training alone (non-SPs). METHODS Twenty-one graduate health professions students participated as health care providers in an interprofessional care simulation involving an SP, caregiver, and clinicians. Six observers in each group received frame-of-reference training and rated aspects of collaborative care using a behavioral observation checklist. The authors examined sources of measurement variance using generalizability theory and extended this technique to statistically compare the rater types and compute reliability for subsets of raters. RESULTS Standardized patient ratings were significantly more reliable than non-SPs' despite both groups receiving extensive rater training. A single SP was predicted to generate scores with a reliability of 0.74, whereas a single non-SP rater's scores were predicted at a reliability of 0.40. Removing each rater one by one from the full 6-member SP sample reduced reliability similarly for all raters (reliability, 0.86-0.89). However, removing individual raters from the full 6-member non-SP sample led to more variable reductions in reliability (0.58-0.72). CONCLUSIONS Ongoing experience rating performance from within a particular simulation-based assessment may be a valuable rater characteristic and more effective than rater training alone. The extensions of reliability estimation introduced here can also be used to support more insightful reliability research and subsequent improvement of rater training and assessment protocols.
منابع مشابه
Assessment of the interprofessional collaboration of healthcare team members: validation of Interprofessional Collaborator Assessment Rubric (ICAR) and pilot study
Background and Aim: Developing Interprofessional education and collaboration is important in systems of medical sciences to deliver better health care. The present study aimed to assess the validation of the Interprofessional Collaborator Assessment Rubric (ICAR) (validity, reliability and confirmatory factor analysis). Also, the interprofessional collaboration of the healthcare team members in...
متن کاملContextualization and validation of the interprofessional collaborator assessment rubric (ICAR) through simulation: Pilot investigation
Background: Simulation can be used for educating, evaluating and assessing psychometric properties of an instrument. The aim of this study was to contextualize and assess the validity and reliability of the Interprofessional Collaborative Assessment tool (ICAR) in an Iranian context using simulation. Methods: In this descriptive study, contextualization of the ICAR was assessed throug...
متن کاملEffectiveness of Educational Workshop based on interprofessional approach in Changing Health Science Students’ Attitudes towards Interprofessional Learning and Collaboration.
Introduction: Considering the developments of the third millennium in the field of health, creating the ability of inter-professional collaboration and teamwork among students is one of the most important tasks of educational organizations. The aim of this study was to investigate the effectiveness of the educational workshop based on inter-professional approach to changing the attitudes of hea...
متن کامل“Letter to Editor” Social Skills and Teamwork in New Generation of Nurses
Effective teamwork is a critical element of a health workplace, but there are always challenges to meet the expectations of employers and workers. However, for the service-users in the health care setting is a requirement that they often presume to be in place. The duty of the policy makers, clinicians, and health system managers is to seek ways of improving the work conditions for workers and ...
متن کاملContextualization and validation of the interprofessional collaborator assessment rubric (ICAR) through simulation: Pilot investigation
BACKGROUND Simulation can be used for educating, evaluating and assessing psychometric properties of an instrument. The aim of this study was to contextualize and assess the validity and reliability of the Interprofessional Collaborative Assessment tool (ICAR) in an Iranian context using simulation. METHODS In this descriptive study, contextualization of the ICAR was assessed through several ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Simulation in healthcare : journal of the Society for Simulation in Healthcare
دوره 10 4 شماره
صفحات -
تاریخ انتشار 2015